Objective

“In stock market, technical analysis is the study of market action, primarily through the use of charts, for the purpose of forecasting future price trends. In its purest form, technical analysis considers only the actual price behavior of the market or instrument, based on the premise that price reflects all relevant factors before an investor becomes aware of them through other channels. Technicians say that a market’s price reflects all relevant information, so their analysis looks more at”internals" than at “externals” such as news events. Price action also tends to repeat itself because investors collectively tend toward patterned behavior - hence technicians’ focus on identifiable trends and conditions." (source: stock2own.com)

In this project I want to analyze relationship between technical indicators, such as moving averages and MACD, and makret price of a stock. In other words, I want to validate the premise that future price of a stock can be predicted by technical indicators. In particular, I’m interested to discover a correlation between technical indicators and a short term (1-2 weeks) price moves.

Data Set

The data set used for this project, contains the list of most US stocks traded on US Stock Exchanges as of December 31st, 2015. List of stocks is limited and includes only stocks that close market day of December 31st, 2015 with a price of $10 per share or higher. It filters out “penny stocks”, which are extremely risky and therefore more volatile than the rest of the market.

The data includes some basic fundamental characteristics as well as prices and technical indicators as of market close on December 31st, 2015. All fundamental characteristics provided as of latest available financial statement.

Last 4 columns show prices as of market close on the first two Fridays of 2016, i.e. as of January 7th and January 14th of 2016. It allows to compute a short term price moves in a form of weekly gain or loss for the first two weeks of 2016.

Data set described in the S2O_StockList.rd file and has the following structure:

## 'data.frame':    2831 obs. of  53 variables:
##  $ InstrID               : int  1064 232480 192339 228611 222723 45497 228365 2431 60 32 ...
##  $ CountryCode           : Factor w/ 1 level "US": 1 1 1 1 1 1 1 1 1 1 ...
##  $ Identifier            : Factor w/ 2831 levels "A","AAC","AACAY",..: 1 2 3 4 5 6 7 8 9 10 ...
##  $ Name                  : Factor w/ 2831 levels "1st Constitution Bancorp",..: 56 8 9 131 111 12 191 10 43 188 ...
##  $ Exchange              : Factor w/ 5 levels "AMEX","NASDAQ",..: 3 3 5 2 1 3 2 2 3 2 ...
##  $ Sector                : Factor w/ 10 levels "","Basic Materials",..: 6 6 3 10 10 8 7 7 8 4 ...
##  $ SPDRIndx              : Factor w/ 10 levels "","XLB","XLE",..: 9 1 1 8 1 1 1 1 10 6 ...
##  $ MktCapType            : Factor w/ 5 levels "","Large","Micro",..: 2 5 2 2 3 4 5 4 2 2 ...
##  $ FS_AsOfDate           : Factor w/ 12 levels "1/31/2015","10/31/2015",..: 2 12 4 12 12 12 12 12 2 12 ...
##  $ FS_BVPS               : Factor w/ 1930 levels "-0.02","-0.03",..: 282 86 151 1704 1153 553 1739 1159 1043 911 ...
##  $ FS_EPS                : num  1.2 0.61 3.62 6.96 10.06 ...
##  $ FS_DivRate            : Factor w/ 328 levels "0","0.01","0.02",..: 47 328 122 41 1 11 328 23 25 197 ...
##  $ FS_GrowthGrade        : num  0 79.8 81.3 53.7 0 ...
##  $ FS_GrowthGradeYOY     : num  7.37 70.41 68.24 27.02 0 ...
##  $ Ratio_PE              : num  33.8 31.25 17.96 5.91 1.71 ...
##  $ Ratio_Beta            : Factor w/ 330 levels "-0.01","-0.02",..: 178 229 90 320 325 58 330 157 134 155 ...
##  $ Ratio_PEG             : Factor w/ 2174 levels "-1","-1.45","-2.99",..: 1 239 624 5 5 1 616 901 1168 415 ...
##  $ Ratio_PEGY            : Factor w/ 2121 levels "-1","0","0.0013",..: 1 251 625 2 2 1 650 941 1282 402 ...
##  $ Ratio_Current         : Factor w/ 2057 levels "0","0.0238","0.0294",..: 1791 1902 1 299 45 452 1378 1636 610 472 ...
##  $ Ratio_Quick           : Factor w/ 2028 levels "0","0.0015","0.0017",..: 1830 1966 1 111 1901 71 1209 1474 68 525 ...
##  $ Ratio_DebtToFCF       : Factor w/ 2216 levels "-0.0001","-0.0002",..: 1716 175 503 475 503 426 57 562 1367 773 ...
##  $ Ratio_ROIC            : Factor w/ 1624 levels "-0.0007","-0.0015",..: 546 766 1426 674 104 469 295 1467 1258 1528 ...
##  $ Eval_ValuePrice_M     : num  6.11 38.88 576.43 419.76 163.94 ...
##  $ Eval_IRT_M            : num  29.01 9.2 6.7 2.81 1.09 ...
##  $ Eval_GrahamNumber     : Factor w/ 2654 levels "0","0.2174","0.8811",..: 723 1838 1439 1618 1324 1292 2609 2406 2317 2355 ...
##  $ MktPrice_Date         : Factor w/ 1 level "12/31/2015": 1 1 1 1 1 1 1 1 1 1 ...
##  $ MktPrice_Close        : num  41.8 19.1 65 42.4 17.2 ...
##  $ MktPrice_Change       : num  -0.36 0.51 0.4 -0.45 -0.39 -0.16 -0.64 -0.45 -0.89 -2.06 ...
##  $ MktPrice_Volume       : int  1449300 143900 1900 6788900 30700 549200 103200 211100 758300 40635300 ...
##  $ MktPrice_VolumeAvg    : int  3294750 243523 10177 7282623 56600 725513 245447 180120 1216273 39398917 ...
##  $ MktPrice_52Wk_Lo      : num  34.7 19.9 48.1 37.6 11.3 ...
##  $ MktPrice_52Wk_Hi      : num  43 44.7 73 55.5 327.2 ...
##  $ MA10                  : num  41.5 19.9 65.2 42.8 17.2 ...
##  $ MA10_ChangePcnt       : num  0.0009 -0.0172 -0.0018 -0.002 0.0171 -0.002 0.0014 -0.0007 -0.0068 -0.0056 ...
##  $ MA30                  : num  40.9 22.1 68.4 42.8 15.1 ...
##  $ MA30_ChangePcnt       : num  0.0028 -0.0052 -0.0031 0 0.0102 -0.0017 0.0009 -0.0006 -0.0029 -0.0025 ...
##  $ MA50                  : num  39.6 22.8 67.1 43.8 18.6 ...
##  $ MA50_ChangePcnt       : num  0.0028 -0.0078 0.0012 -0.0011 -0.012 -0.0125 -0.0012 0.0018 -0.0051 -0.0015 ...
##  $ MA150                 : num  38.5 28.8 60.4 42.1 67.1 ...
##  $ MA150_ChangePcnt      : num  0.0001 -0.0046 0.001 0 -0.015 -0.0025 -0.0002 -0.0001 -0.0001 -0.0014 ...
##  $ MA200                 : num  39.4 29.8 60.5 43.8 102.4 ...
##  $ MA200_ChangePcnt      : num  0 -0.0018 -0.0003 -0.0013 -0.0094 -0.0009 0.0011 0.0002 0 -0.001 ...
##  $ RSI                   : num  56 30.7 38.6 43.1 52.4 ...
##  $ RSI_ChangePcnt        : num  -0.1051 0.5708 0.1799 -0.0913 -0.0416 ...
##  $ SlowStoch_K           : num  0.8579 0.0814 0.0977 0.5506 0.7596 ...
##  $ SlowStoch_D           : num  0.88 0.1059 0.0846 0.5279 0.8556 ...
##  $ SlowStoch_ChangePcnt  : num  -0.0214 0.3523 0.9004 -0.0312 -0.0356 ...
##  $ MACD                  : num  0.0299 -0.2173 -0.0358 -0.0004 0.3963 ...
##  $ MACD_ChangePcnt       : num  -0.636 0.363 0.699 -1.005 -0.386 ...
##  $ Jan_07_MktPrice_Close : num  39 18.2 58.1 40.5 15.5 ...
##  $ Jan_07_MktPrice_Volume: int  3502100 97600 2600 11282600 94400 891200 372100 152900 1342600 80229800 ...
##  $ Jan_14_MktPrice_Close : num  37.6 17.3 57.3 40.5 17 ...
##  $ Jan_14_MktPrice_Volume: int  2893300 210600 15500 11714900 26300 643900 259100 126600 1109900 62424200 ...

Data Summary

##     InstrID       CountryCode   Identifier  
##  Min.   :    13   US:2831     A      :   1  
##  1st Qu.:  1880               AAC    :   1  
##  Median :  4662               AACAY  :   1  
##  Mean   : 60429               AAL    :   1  
##  3rd Qu.: 85768               AAMC   :   1  
##  Max.   :247891               AAN    :   1  
##                               (Other):2825  
##                            Name             Exchange   
##  1st Constitution Bancorp    :   1   AMEX       :  34  
##  1st Source Corporation      :   1   NASDAQ     :1119  
##  3M Company                  :   1   NYSE       :1431  
##  51job Inc.                  :   1   OTC BB     :   1  
##  A-Mark Precious Metals, Inc.:   1   OTC Markets: 246  
##  A. Schulman, Inc.           :   1                     
##  (Other)                     :2825                     
##               Sector       SPDRIndx    MktCapType       FS_AsOfDate  
##  Financial       :698          :2382        :   3   9/30/2015 :1899  
##  Services        :457   XLF    :  84   Large:1013   12/31/2014: 442  
##  Technology      :354   XLY    :  83   Micro: 104   10/31/2015: 239  
##  Industrial Goods:261   XLK    :  65   Mid  :1291   3/31/2015 :  83  
##  Consumer Goods  :246   XLI    :  62   Small: 420   8/31/2015 :  63  
##  Healthcare      :184   XLV    :  49                6/30/2015 :  58  
##  (Other)         :631   (Other): 106                (Other)   :  47  
##     FS_BVPS         FS_EPS           FS_DivRate   FS_GrowthGrade  
##  NULL   :  66   Min.   :    0.01   NULL   : 751   Min.   :  0.00  
##  12.34  :   7   1st Qu.:    0.88   0.4    :  59   1st Qu.: 20.52  
##  0      :   6   Median :    1.63   1      :  56   Median : 41.98  
##  9.33   :   6   Mean   :   12.55   0.24   :  53   Mean   : 41.89  
##  10.35  :   5   3rd Qu.:    2.89   1.2    :  53   3rd Qu.: 62.51  
##  11.64  :   5   Max.   :13852.27   0.6    :  47   Max.   :100.00  
##  (Other):2736                      (Other):1812                   
##  FS_GrowthGradeYOY    Ratio_PE           Ratio_Beta     Ratio_PEG   
##  Min.   : 0.00     Min.   :   0.5337   NULL   : 195   -1     : 447  
##  1st Qu.:20.08     1st Qu.:  13.9960   0      :  36   NULL   :  76  
##  Median :32.81     Median :  19.5133   0.87   :  29   0      :  71  
##  Mean   :33.12     Mean   :  37.7833   0.91   :  29   0.0805 :   3  
##  3rd Qu.:44.99     3rd Qu.:  30.5931   1.28   :  28   0.3299 :   3  
##  Max.   :93.90     Max.   :1793.6667   0.96   :  27   0.5575 :   3  
##                                        (Other):2487   (Other):2228  
##    Ratio_PEGY   Ratio_Current   Ratio_Quick    Ratio_DebtToFCF
##  -1     : 446   NULL   : 612   NULL   : 617   0        : 292  
##  0      : 128   0      :  80   0      :  94   NULL     : 135  
##  NULL   :  53   11.1351:   4   0.6529 :   3   6714     :   9  
##  0.1386 :   3   1      :   3   0.697  :   3   2517.75  :   8  
##  0.2368 :   3   1.2343 :   3   0.8781 :   3   5035.5   :   8  
##  0.4893 :   3   0.2172 :   2   0.9288 :   3   1831.0909:   7  
##  (Other):2195   (Other):2127   (Other):2108   (Other)  :2372  
##    Ratio_ROIC   Eval_ValuePrice_M    Eval_IRT_M     Eval_GrahamNumber
##  NULL   :  47   Min.   :    0.00   Min.   : 0.000   NULL   : 103     
##  0.0783 :   8   1st Qu.:    6.86   1st Qu.: 8.366   0      :   7     
##  0.0916 :   8   Median :   16.36   Median :11.076   26.7836:   4     
##  0.0513 :   7   Mean   :  118.20   Mean   :14.209   13.4363:   3     
##  0.06   :   7   3rd Qu.:   39.68   3rd Qu.:15.307   13.9862:   3     
##  0.0683 :   7   Max.   :97133.20   Max.   :99.000   15.2403:   3     
##  (Other):2747                                       (Other):2708     
##     MktPrice_Date  MktPrice_Close      MktPrice_Change    
##  12/31/2015:2831   Min.   :    10.02   Min.   :-2281.000  
##                    1st Qu.:    18.63   1st Qu.:   -0.630  
##                    Median :    30.57   Median :   -0.280  
##                    Mean   :   189.47   Mean   :   -2.013  
##                    3rd Qu.:    53.58   3rd Qu.:   -0.030  
##                    Max.   :197800.00   Max.   :    8.330  
##                                                           
##  MktPrice_Volume    MktPrice_VolumeAvg MktPrice_52Wk_Lo   
##  Min.   :       0   Min.   :     107   Min.   :     1.00  
##  1st Qu.:   51950   1st Qu.:   63996   1st Qu.:    15.62  
##  Median :  232400   Median :  281887   Median :    26.03  
##  Mean   :  766610   Mean   : 1076697   Mean   :   111.01  
##  3rd Qu.:  728400   3rd Qu.:  970222   3rd Qu.:    45.03  
##  Max.   :46594300   Max.   :71855807   Max.   :196005.00  
##                                                           
##  MktPrice_52Wk_Hi         MA10           MA10_ChangePcnt     
##  Min.   :    10.38   Min.   :     9.42   Min.   :-0.0400000  
##  1st Qu.:    24.57   1st Qu.:    18.64   1st Qu.:-0.0028000  
##  Median :    39.58   Median :    30.65   Median :-0.0007000  
##  Mean   :   233.33   Mean   :   190.50   Mean   :-0.0003408  
##  3rd Qu.:    66.81   3rd Qu.:    53.42   3rd Qu.: 0.0017000  
##  Max.   :224675.00   Max.   :199114.50   Max.   : 0.0783000  
##                                                              
##       MA30           MA30_ChangePcnt           MA50          
##  Min.   :     8.44   Min.   :-0.0302000   Min.   :     7.55  
##  1st Qu.:    18.80   1st Qu.:-0.0019000   1st Qu.:    18.98  
##  Median :    31.05   Median :-0.0003000   Median :    31.35  
##  Mean   :   192.22   Mean   :-0.0003416   Mean   :   193.08  
##  3rd Qu.:    54.04   3rd Qu.: 0.0011000   3rd Qu.:    54.20  
##  Max.   :200910.60   Max.   : 0.0405000   Max.   :201946.90  
##                                                              
##  MA50_ChangePcnt          MA150           MA150_ChangePcnt    
##  Min.   :-0.0398000   Min.   :     2.71   Min.   :-0.0426000  
##  1st Qu.:-0.0017000   1st Qu.:    19.50   1st Qu.:-0.0013000  
##  Median :-0.0002000   Median :    32.28   Median :-0.0003000  
##  Mean   :-0.0004707   Mean   :   195.32   Mean   :-0.0004859  
##  3rd Qu.: 0.0010000   3rd Qu.:    54.91   3rd Qu.: 0.0005000  
##  Max.   : 0.0345000   Max.   :204735.60   Max.   : 0.0345000  
##                                                               
##      MA200           MA200_ChangePcnt         RSI       
##  Min.   :     2.65   Min.   :-0.262400   Min.   : 0.00  
##  1st Qu.:    19.91   1st Qu.:-0.001000   1st Qu.:39.16  
##  Median :    32.81   Median :-0.000200   Median :46.19  
##  Mean   :   198.71   Mean   :-0.000392   Mean   :47.36  
##  3rd Qu.:    55.55   3rd Qu.: 0.000400   3rd Qu.:54.34  
##  Max.   :207714.80   Max.   : 0.027800   Max.   :96.45  
##                                                         
##  RSI_ChangePcnt      SlowStoch_K      SlowStoch_D     SlowStoch_ChangePcnt
##  Min.   :-0.50520   Min.   :0.0000   Min.   :0.0118   Min.   :-1.00000    
##  1st Qu.:-0.18300   1st Qu.:0.4361   1st Qu.:0.4326   1st Qu.:-0.11142    
##  Median :-0.10590   Median :0.6414   Median :0.6187   Median :-0.04661    
##  Mean   :-0.08466   Mean   :0.6016   Mean   :0.6010   Mean   :-0.02585    
##  3rd Qu.:-0.01255   3rd Qu.:0.7893   3rd Qu.:0.7979   3rd Qu.: 0.02553    
##  Max.   : 1.67530   Max.   :1.0000   Max.   :1.0000   Max.   : 5.88235    
##                                                                           
##       MACD          MACD_ChangePcnt     Jan_07_MktPrice_Close
##  Min.   :-11.0949   Min.   :-132.7540   Min.   :     8.18    
##  1st Qu.: -0.0016   1st Qu.:  -0.6324   1st Qu.:    17.75    
##  Median :  0.0581   Median :  -0.3252   Median :    28.76    
##  Mean   :  0.1268   Mean   :  -0.6957   Mean   :   185.57    
##  3rd Qu.:  0.1549   3rd Qu.:  -0.0961   3rd Qu.:    50.70    
##  Max.   : 57.6074   Max.   : 155.9407   Max.   :195580.00    
##                                                              
##  Jan_07_MktPrice_Volume Jan_14_MktPrice_Close Jan_14_MktPrice_Volume
##  Min.   :      100      Min.   :     7.27     Min.   :      100     
##  1st Qu.:    66200      1st Qu.:    17.25     1st Qu.:    69200     
##  Median :   341700      Median :    28.05     Median :   354100     
##  Mean   :  1568085      Mean   :   182.19     Mean   :  1635207     
##  3rd Qu.:  1336050      3rd Qu.:    49.48     3rd Qu.:  1362000     
##  Max.   :115361300      Max.   :192250.00     Max.   :125143800     
##

Data Overview

Each stock characterized by it’s price and trading volume. Trading volume shows how many shares have changed hands (were bought and sold) during the trading day. Volume shows how liquid the stock is. In order to make a proper conclusions we should focus on liquid stocks, those that trade a lot every day.

##      Min.   1st Qu.    Median      Mean   3rd Qu.      Max. 
##     10.02     18.63     30.57    189.50     53.58 197800.00
##     Min.  1st Qu.   Median     Mean  3rd Qu.     Max. 
##      107    64000   281900  1077000   970200 71860000

Price ranges from $10.02 to $197,800 per share and avergae volume is in range from 107 to 71 mln shares.

The variety is enormous, therefore a logarithmic scale seems appropriate for the plot.

Extremely highly priced stocks have relatively small trading volume - perhaps not so many people can buy a stock for $197,000 per share. It was expected.

Introducing a new variable: VAvg_Mln - average daily volume in millions. Now it is a bit easier to work with extremely large volumes.

Average daily volume vs market price (price is limited by 99% quantile):

Very few stocks that priced above $100 have daily volume greater than 20 million shares.

Price variety by sector (99% quantile):

For most of the sectors the median price per share is below $50 and Healthcare sector seems to have most expensive stocks. At the same time, Financial sector has more outliers with higher stock prices.

Volume variety by sector (99% quantile):

Basic Materials stocks are among most havily traded stocks, which is probably explainable by the recent crash in oil prices.

In order to minimize bias, it seems reasonable to exclude less liquid stocks from the analysis. We want to get rid of the 1st quantile of data and put a limit of 64,000 shares as a minimum accepted value for an average trading volume. Let’s see how this may affect overall stocks diversity by two major classifications: market capitalization and sector. Note though, this rule will eliminate outliers on the lower side only, therefore, in order to make it easier to compare plots, we shall also limit below plots by 99% percentile on y axis.

It definitely changed the picture - it almost wiped out market capitalization category “micro” and significantly reduced “small” stocks population. It also eliminated big outliers with prices in $100,000 range. And both results are satisfying - we want to focus analysis on a liquid stocks and micro and small cap stocks as well as extremely highly priced stocks with small daily trading volumes should be excluded.

Sector presence seems OK and stocks that do not have sector classification are among those that disappeared the most from the plot. Perhaps not liquid stocks do not have as good data quality as those that traded a lot.

Let’s check how market capitalization affect stock price and trading volume on a limited data set.

It is clear that the larger the market capitalization, the bigger the trade volume is. There are some large stocks could be found with a relatively small price (around $10), however, it is really rare to see mideum or small cap stocks traded with a price significantly greater than $100.

The correlation between market price and average volume is rather small - the red linear correlation line has relatively small positive slop.

The similar plots by sectors:

Data Transformations

The objective of this project is to discover relationship between technical indicators and future stock price change. Each group of indicators usually used in a certain specific way by market technitians. In this project we will focus on Moving Averages, RSI and MACD.

Moving Averages (MA)

Data set contains information about 5 moving averages:

  • short term moving averages: MA10 and MA30 (10 and 30 days moving averages)
  • medium term: MA50
  • long term: MA150 and MA200

In general we are not interested in absolute values of the MA, instead technical analysts are focusing on moving average relative to the price line or to other MAs. In other words, in order to identify the trend we want to know if the price is above or below the moving average and how moving averages are positioned against each other (above, below, crossed over). In particular, it is usually advised to compare moving averages of different time frames. For example, check when a short term moving average (MA10 or MA30) has crossed medium term (MA50) and long term (MA150 and MA200) moving averages. (source: stock2own.com)

MAx_Price columns will show the distance in % between MA and the price.

RSI

“The RSI is classified as a momentum oscillator, measuring the velocity and magnitude of directional price movements. Momentum is the rate of the rise or fall in price. … The RSI is most typically used on a 14 day timeframe, measured on a scale from 0 to 100.

Traditionally, RSI readings greater than the 70 level are considered to be in overbought territory, and RSI readings lower than the 30 level are considered to be in oversold territory. In between the 30 and 70 level is considered neutral, with the 50 level a sign of no trend." (source: stock2own.com)

Therefore we will compute RSIV as a difference between RSI and the 50 level.

MACD

“The MACD-Histogram represents the difference between the MACD and its trigger line, the 9-day EMA of MACD. The plot of this difference is presented as a histogram, making centerline crossovers and divergences easily identifiable. A centerline crossover for the MACD-Histogram is the same as a moving average crossover for MACD.” (source: stock2own.com)

The MACD value in a dataset is actually a MACD histogram value, therefore no modifications required for MACD.

Weekly Price Change

Finally, we have to compute weekly price change for the first two weeks of January. Because we have technical indicators as of December 31st, it makes sense to compute price change for the first and second weeks as well as price change for two complete weeks. We can think about it as a short term (1 week) and a little bit longer term (2 weeks) projections. All changes computed as a percentage, so we can easily compare them for stocks with significantly different prices.

Final Recordset

Let’s get rid of all unneccessary data, i.e. exclude low volume stocks and columns that we are not going to analyze. Also we shall rename column names to make them shorter and easier to use.

Note though that all values describing price, volume or technical indicators in this final dataset are not absolute values - they are percent of change of a value or percent of the difference between two values.

Data Set Structure

## 'data.frame':    2123 obs. of  28 variables:
##  $ InstrID     : int  1064 232480 228611 45497 228365 2431 60 32 69684 2370 ...
##  $ Sector      : Factor w/ 10 levels "","Basic Materials",..: 6 6 10 8 7 7 8 4 5 8 ...
##  $ MktCap      : Factor w/ 5 levels "","Large","Micro",..: 2 5 2 4 5 4 2 2 4 4 ...
##  $ EPS         : num  1.2 0.61 6.96 1.87 0.54 0.79 6.81 9.22 0.85 3.47 ...
##  $ GrGrade     : num  0 79.8 53.7 40.1 59.8 ...
##  $ GrGradeYOY  : num  7.37 70.41 27.02 48.43 50.44 ...
##  $ MA10_Price  : num  0.0074 -0.0456 -0.0102 -0.0089 -0.0082 -0.0017 0.0014 -0.0207 -0.0104 -0.0097 ...
##  $ MA30_Price  : num  0.0215 -0.16 -0.0113 -0.0389 -0.0245 -0.0245 -0.0322 -0.078 -0.0209 0.0135 ...
##  $ MA50_Price  : num  0.0536 -0.1973 -0.0331 -0.1385 -0.0472 ...
##  $ MA150_Price : num  0.0787 -0.5089 0.0066 -0.4801 -0.0921 ...
##  $ MA200_Price : num  0.0569 -0.563 -0.0338 -0.464 -0.0309 ...
##  $ MA10_MA50   : num  0.0465 -0.145 -0.0227 -0.1284 -0.0387 ...
##  $ MA10_MA200  : num  0.0499 -0.4947 -0.0234 -0.4511 -0.0225 ...
##  $ MA50_MA200  : num  0.0035 -0.3054 -0.0007 -0.286 0.0156 ...
##  $ RSI         : num  0.12 -0.386 -0.139 -0.177 -0.123 ...
##  $ MACD        : num  0.0299 -0.2173 -0.0004 0.147 0.0986 ...
##  $ Price       : num  -0.0086 0.0268 -0.0106 -0.0071 -0.0373 -0.0194 -0.0059 -0.0196 -0.0193 0.0087 ...
##  $ Volume      : num  -0.5601 -0.4091 -0.0678 -0.243 -0.5795 ...
##  $ Wk1_Price   : num  -0.0672 -0.0451 -0.0449 -0.0768 -0.109 ...
##  $ Wk1_Volume  : num  0.0629 -0.5992 0.5492 0.2284 0.516 ...
##  $ Wk2_Price   : num  -0.0356 -0.0511 0.0025 0.0169 -0.0739 -0.058 -0.0505 0.0318 -0.0216 -0.0512 ...
##  $ Wk2_Volume  : num  -0.1218 -0.1352 0.6086 -0.1125 0.0556 ...
##  $ Wk_2_Price  : num  -0.1005 -0.0939 -0.0425 -0.0612 -0.1748 ...
##  $ MA10_ChPcnt : num  0.0009 -0.0172 -0.002 -0.002 0.0014 -0.0007 -0.0068 -0.0056 -0.0034 0.0025 ...
##  $ MA30_ChPcnt : num  0.0028 -0.0052 0 -0.0017 0.0009 -0.0006 -0.0029 -0.0025 -0.0008 0.0009 ...
##  $ MA50_ChPcnt : num  0.0028 -0.0078 -0.0011 -0.0125 -0.0012 0.0018 -0.0051 -0.0015 -0.0025 0.0003 ...
##  $ MA150_ChPcnt: num  0.0001 -0.0046 0 -0.0025 -0.0002 -0.0001 -0.0001 -0.0014 -0.0002 -0.002 ...
##  $ MA200_ChPcnt: num  0 -0.0018 -0.0013 -0.0009 0.0011 0.0002 0 -0.001 -0.0006 -0.0006 ...

Data Summary

##     InstrID                    Sector      MktCap          EPS        
##  Min.   :    13   Financial       :443        :   3   Min.   : 0.010  
##  1st Qu.:  1497   Services        :390   Large: 826   1st Qu.: 0.895  
##  Median :  3441   Technology      :285   Micro:   1   Median : 1.710  
##  Mean   : 52076   Industrial Goods:227   Mid  :1143   Mean   : 2.489  
##  3rd Qu.: 53906   Consumer Goods  :201   Small: 150   3rd Qu.: 3.010  
##  Max.   :247891   Healthcare      :170                Max.   :49.900  
##                   (Other)         :407                                
##     GrGrade         GrGradeYOY      MA10_Price          MA30_Price      
##  Min.   :  0.00   Min.   : 0.00   Min.   :-0.270500   Min.   :-0.37490  
##  1st Qu.: 23.01   1st Qu.:21.63   1st Qu.:-0.012050   1st Qu.:-0.04220  
##  Median : 44.72   Median :34.70   Median :-0.003200   Median :-0.01420  
##  Mean   : 44.18   Mean   :34.81   Mean   :-0.002161   Mean   :-0.01646  
##  3rd Qu.: 65.25   3rd Qu.:47.19   3rd Qu.: 0.005900   3rd Qu.: 0.01015  
##  Max.   :100.00   Max.   :93.90   Max.   : 0.136600   Max.   : 0.42740  
##                                                                         
##    MA50_Price        MA150_Price        MA200_Price      
##  Min.   :-0.57640   Min.   :-1.59810   Min.   :-1.94760  
##  1st Qu.:-0.05735   1st Qu.:-0.11415   1st Qu.:-0.14365  
##  Median :-0.01850   Median :-0.02470   Median :-0.03430  
##  Mean   :-0.02469   Mean   :-0.05394   Mean   :-0.07308  
##  3rd Qu.: 0.01355   3rd Qu.: 0.04345   3rd Qu.: 0.04480  
##  Max.   : 0.49600   Max.   : 0.80000   Max.   : 0.80440  
##                                                          
##    MA10_MA50          MA10_MA200         MA50_MA200      
##  Min.   :-0.65310   Min.   :-1.93910   Min.   :-1.99660  
##  1st Qu.:-0.05120   1st Qu.:-0.14125   1st Qu.:-0.10470  
##  Median :-0.01600   Median :-0.03090   Median :-0.01440  
##  Mean   :-0.02254   Mean   :-0.07129   Mean   :-0.04374  
##  3rd Qu.: 0.01380   3rd Qu.: 0.04600   3rd Qu.: 0.04260  
##  Max.   : 0.45690   Max.   : 0.80960   Max.   : 0.71260  
##                                                          
##       RSI                MACD              Price          
##  Min.   :-1.00000   Min.   :-3.28040   Min.   :-0.106800  
##  1st Qu.:-0.21910   1st Qu.: 0.00000   1st Qu.:-0.016000  
##  Median :-0.09000   Median : 0.06550   Median :-0.009900  
##  Mean   :-0.06985   Mean   : 0.09791   Mean   :-0.008403  
##  3rd Qu.: 0.05980   3rd Qu.: 0.16720   3rd Qu.:-0.003100  
##  Max.   : 0.83260   Max.   : 2.41180   Max.   : 0.091700  
##                                                           
##      Volume           Wk1_Price          Wk1_Volume     
##  Min.   :-0.99880   Min.   :-0.52610   Min.   :-0.9998  
##  1st Qu.:-0.41065   1st Qu.:-0.08110   1st Qu.:-0.0299  
##  Median :-0.25190   Median :-0.05400   Median : 0.2674  
##  Mean   :-0.16133   Mean   :-0.05492   Mean   : 0.3929  
##  3rd Qu.:-0.05035   3rd Qu.:-0.02685   3rd Qu.: 0.6189  
##  Max.   : 9.37710   Max.   : 0.19020   Max.   :15.9551  
##                                                         
##    Wk2_Price          Wk2_Volume        Wk_2_Price      
##  Min.   :-0.33680   Min.   :-0.9372   Min.   :-0.51150  
##  1st Qu.:-0.05320   1st Qu.:-0.0454   1st Qu.:-0.11690  
##  Median :-0.02550   Median : 0.2779   Median :-0.07840  
##  Mean   :-0.02949   Mean   : 0.4754   Mean   :-0.08259  
##  3rd Qu.:-0.00240   3rd Qu.: 0.6949   3rd Qu.:-0.04160  
##  Max.   : 0.48290   Max.   :25.4154   Max.   : 0.37260  
##                                                         
##   MA10_ChPcnt          MA30_ChPcnt          MA50_ChPcnt        
##  Min.   :-0.0400000   Min.   :-0.0302000   Min.   :-0.0215000  
##  1st Qu.:-0.0030000   1st Qu.:-0.0020000   1st Qu.:-0.0018000  
##  Median :-0.0010000   Median :-0.0004000   Median :-0.0003000  
##  Mean   :-0.0006314   Mean   :-0.0004065   Mean   :-0.0005701  
##  3rd Qu.: 0.0012000   3rd Qu.: 0.0011000   3rd Qu.: 0.0010000  
##  Max.   : 0.0783000   Max.   : 0.0405000   Max.   : 0.0345000  
##                                                                
##   MA150_ChPcnt         MA200_ChPcnt       
##  Min.   :-0.0426000   Min.   :-0.0074000  
##  1st Qu.:-0.0014000   1st Qu.:-0.0011000  
##  Median :-0.0004000   Median :-0.0003000  
##  Mean   :-0.0005529   Mean   :-0.0003563  
##  3rd Qu.: 0.0004000   3rd Qu.: 0.0004000  
##  Max.   : 0.0345000   Max.   : 0.0278000  
##

Analysis

Price changes for the first two weeks of January 2016

For the first week of January most of the stock prices moved down and the next week the trend continues, but with the lower pace - more price changes were close to 0 or even ended the week in a slightly positive territory.

It is easy to compare price changes between week 1 and week 2 on the following plots.

We can also see that volume of trades on week 2 was slightly higher with a few really big moves - for some stocks the volume increased 20 times compare to the average daily volume.

Let’s discover this trend further and take a close look to the majority of stocks volume changes:

In general, the volume keep growing with a similar pace (we show volume changes on the plot, therefore positive values mean positive growth). At the same time the number of outliers with really big volume increase has grown on week 2.

Similar analysis for the price change by sector clearly shows that median price change is getting closer to a 0 level on week 2, which means that situation on a market is stabilazing a bit. Basic materials and Utilities are among those sectors that recovered their losses the most. Once again, it seems explainable by fluctuation of the oil prices.

The price changes for the week 1 and week 2 by sector show that price changes for the second week of January is less dramatic (less spread) and moving close to a zero line and to the positive territory.

Cumulative two weeks price change by sector:

Before we start analyzing dependency between technical indicators and price changes, it is interesting to take a look to the technical indicator values distribution. Note that moving average values here show the difference between stock price as of December 31st, 2015 and the corresponding moving average value. For example, MA10_Price of 10% means that 10 day moving average is above the price by 10%.

The longer the moving average time frame, the wider changes we can see, which is expected, because a 1 day moving average will follow the price line really closely. Based on these plots, most of the moving averages have negative values with median slightly below 0 line. Perhaps, it means that the majority of stocks have been in the “negative” territory for quite a while.

The plot for RSI is somewhat similar. Because of the computation we made for RSI, we should keep in mind that anything below zero means that those stocks “undeperform”, while all positive values show “well performed”, compare to their previous performance, stocks.

MACD shows slightly differnet picture - majority of the stocks are really close to a 0 line, with larger number of stocks in a positive territory. Similarly to the MA plots, MACD plot also has a long tails.

Correlation between technical indicators and price changes

We need to transform stocks data set (reshape from “wide” to “long” format) in order to easily create plots between each technical indicator and price change.

Let’s start with a simple scatter plots with added linear correlation line.

Technical indicators vs first week price change

The longer the term of the moving average, the bigger spread of the change between MA and price value.

Technical indicators vs second week price change

There is a small positive correlation between moving average and a price change for most of the moving averages. However, the shortest one - 10 days moving average - seems to have a negative correlation. In all cases the linear correlation seems to be really small, insignificant.

Let’s examine numerical values for correlation coefficients using Pearson model as a measure of a possible linear correlation. I’m also interested to see how strong the depency is, therefore I’m adding a Kendall’s rank as well.

Pearson Correlation

Compute Pearson Correlation to measure the linear correlation between technical indicators and the price change.

Indicator Week1 Week2 TwoWeeks
MA10 -0.07114288 -0.02548289 -0.06690046
MA30 0.13005578 0.07487754 0.13844815
MA50 0.17147153 0.10601693 0.18720170
MA150 0.18379466 0.16855280 0.23853028
MA200 0.18794551 0.15898587 0.23498567
MA10_MA50 0.20138433 0.11896733 0.21660820
MA10_MA200 0.19777916 0.16167001 0.24351077
MA50_MA200 0.15149225 0.13944048 0.19771496
RSI 0.03672454 -0.01744844 0.01553864
MACD -0.08038889 -0.02820322 -0.07430319
MA10 change -0.01990002 -0.00019667 -0.01547899
MA30 change 0.17068588 0.05510045 0.15286969
MA50 change 0.12776281 0.10426879 0.15636184
MA150 change 0.15370422 0.08315042 0.16197400
MA200 change 0.12708464 0.05388359 0.12388077

Kendall Rank

Compute Kendall Rank to measure the association between technical indicators and price changes.

Indicator Week1 Week2 TwoWeeks
MA10 -0.07114288 -0.02548289 -0.06690046
MA30 0.13005578 0.07487754 0.13844815
MA50 0.17147153 0.10601693 0.18720170
MA150 0.18379466 0.16855280 0.23853028
MA200 0.18794551 0.15898587 0.23498567
MA10_MA50 0.20138433 0.11896733 0.21660820
MA10_MA200 0.19777916 0.16167001 0.24351077
MA50_MA200 0.15149225 0.13944048 0.19771496
RSI 0.03672454 -0.01744844 0.01553864
MACD -0.08038889 -0.02820322 -0.07430319
MA10 change -0.01990002 -0.00019667 -0.01547899
MA30 change 0.17068588 0.05510045 0.15286969
MA50 change 0.12776281 0.10426879 0.15636184
MA150 change 0.15370422 0.08315042 0.16197400
MA200 change 0.12708464 0.05388359 0.12388077

Interestingly enough, the highest correlation coefficiant values belong to the ratios that describe relationship between moving averages (MA10_MA50, MA10_MA200 and MA50_MA200) and also to the values that describe most recent moving avearge change. The latest one was thrown into the list of interesting parameters because it also gives us an idea of the very short term moving average incline - if the value of “MA10 change” is positive it means that moving average line on a chart is pointed up and when it is negative, the line is pointing down.

The other higher correlation values could be found in the column “TwoWeeks”, which shows correlation between technical indicators and two weeks price change. Interesting that relatively high values are for MA150 and MA200 - 0.2385 and 0.2349. It could be explained because 200 days moving average is so popular that so many investors are using it for longer term projections and we can expect this correlation to grow even more for longer time frames for price change tracking.

However, even the highest values are in the low .20th range, meaning that there is very little correlation found. Let’s see if we can improve the situation by adding fundamental characteristics to our analysis.

Mixing technical and fundamental data together

There are two interesting fundamental characteristics available in the original data set - GrowthGrade and GrowthGradeYOY. As described in the documentation, Growth Grade weights historical growth rate for five major financial metrics (Equity, EPS, Sales, Free Cash Flow, Cash from Operating Activity and ROIC) and year and presented as a float in a range 0..100. So, it could be a nice starting point, where we have data from multiple fundamental metrics aggregated in one easy to use value.

Note, that GrowthGradeYOY is almost the same is GrowthGrade, except it is using year over year growth instead of historical (multiple years growth) figure. Therefore, we can work with the GrowthGradeYOY in the same fasion.

First of all, we want to convert Growth Grade values into factored values. It should allow us easily add it to our analysis and mix it with technical characteristics. A simple way would be to group all values in a buckets and name each of them. Bucket size of 20 seems reasonable. Therefore we have 5 buckets above 0 and we name them as A, B, C, D, E. For the 0 grade we will use F (there are should be very few such stocks). So, the best grade is A and the worst is F.

Now we can redo reshaping and include these new values as a key variables.

Technical indicators vs first week price change with historical Growth Grade

It does not look like adding Growth Grade helped - we can see stocks with positive and negative Growth Grades in each quadrant of the plots, meaning that there are many scenarios where high graded stocks prices are folling even when technical indicators are still in a positive territory.

Final Plots and Summary

The data set used for this project, contains information about 2,831 stocks traded on US Stock Exchanges as of December 31st, 2015 with price per share of $10 or higher. The average volume of daily trade for these stocks is in range from 107 to 71 mln shares.

In order to minimize bias, low liquid stocks with average daily volume of 64,000 shares or less were eliminated from analysis. It significantly reduced presence of micro and small stocks, but kept all market sectors well presented.

The objective of this project is discover a correlation between technical indicators and a short term (1-2 weeks) stocks price changes. The chosen first two weeks of January, 2016 were relatively active for the stock market with a significant down trend - most stock prices were going down for both weeks.

We used short, medium and long term moving averages, RSI and MACD as technical indicators. All absolute values available for the day of December 31st, 2015 were converted to a change percentage. As a result we have tried to find correlation between future 1 and 2 weeks price change and the following values:

In this project we could not identify any significant correlation between described above technical indicator parameters and short-term future price changes. Below are Pearson Correlation coefficients between technical indicators and the price change.

Indicator Week1 Week2 TwoWeeks
MA10 -0.07114288 -0.02548289 -0.06690046
MA30 0.13005578 0.07487754 0.13844815
MA50 0.17147153 0.10601693 0.18720170
MA150 0.18379466 0.16855280 0.23853028
MA200 0.18794551 0.15898587 0.23498567
MA10_MA50 0.20138433 0.11896733 0.21660820
MA10_MA200 0.19777916 0.16167001 0.24351077
MA50_MA200 0.15149225 0.13944048 0.19771496
RSI 0.03672454 -0.01744844 0.01553864
MACD -0.08038889 -0.02820322 -0.07430319
MA10 change -0.01990002 -0.00019667 -0.01547899
MA30 change 0.17068588 0.05510045 0.15286969
MA50 change 0.12776281 0.10426879 0.15636184
MA150 change 0.15370422 0.08315042 0.16197400
MA200 change 0.12708464 0.05388359 0.12388077

The highest correlation rates found are less than 0.25 and belong to the changes in moving average value and to the difference between moving averages of different length. The correlation coefficient is too small to be significant, however it suggests that perhaps a better described moving averages value changes (current and historical) can better predict future change in a corresponding stock price. Mathematical description of the relationship between moving averages of different lengths should also be considered as a possible point of interest for further research.

Other notable observation is MACD and the shortest analyzed moving average - 10 days moving average - have small but negative correlation to future price change, while all other moving averages and RSI have correlation with a positive sign. Perhaps it could be explained by high volatility on stock market and it may be suggested to explore this dependency further, it could be used as an early signal for price direction change.

In a search for additional parameter that could help differentiate stocks and perhaps improve situation, we added fundamental characteristic - Growth Grade, which combines growth rates for five major financial metrics (Equity, EPS, Sales, Free Cash Flow, Cash from Operating Activity and ROIC) for the last 10 years (or as many as available). However, it did not improve results and a typical plot that shows future price change versus a technical indicator looks like the plots for moving averages shown below.

The figures on these plots are almost symmetrical and cover all four quadrants of the plot, meaning that there are relatively similar number of stocks with good fundamental grades could be found in the left section of the plot, where corresponding moving averages are above the price line and in the right section, presenting stocks which moving averages are below the price line. Both those scenarios found in the above and below zero line sections, representing price change of a stock, meaning that the number of good graded stocks that went down is similar to the number of bad rated stocks that went down.

These charts support low correlation coefficiants computed for each analyzed technical indicator. In other words, we could not discover any significant correlation between analyzed representation of technical indicators and short term price changes of a stock.

The results of this research should be considered as a reminder that we live in a time of global economy, where each country and each business depends on everything else. It seems important to keep in mind that during first weeks of 2016 the US stock market was moved mostly by global news: oil prices dropped below 12 year low across the globe; China (the second largest economy) kept sending warning signals about slowing down it’s growth; warries about Greece and Eurozone future and more. All these news practically hid all individual business achievements. It makes possible to think that by choosing a different time frame we could receive different results and if so, such models should be used with extreme caution.

Reflection

I selected stocks information for this research because I’m really interested in stock analysis. I expected a bit better results though, I was really hoping to find a correlation between technical indicators and future price moves. The results confirmed that trading stocks probably more an art than a science. It also gave me the hint where to go for further data exploration and analysis and I’m going to work on additional information for the data set, such as more detailed description of a trend, perhaps in a form of pivot points, for moving averages. I’m curious how that could change results of a similar analysis.

Another interesting experiment would be to run the same type of analysis for few other time frames. May be run it for every quarter of 2015, trying to avoid earnings season, where stock prices become extremely volatile and very news-driven.

I really enjoyed working on this project - R and R-studio are absolutely amazing! I love the whole idea of keeping research notes and code in one .rmd file. One thing that I would like to explore more is how to effectively split my work into multiple files, because the larger the file, the longer time it takes to knit it.